Method and system for emergent data processing

ABSTRACT

A method and system for emergent data processing are described. A system having one or more servers operable to handle retail data can receive content including customer data and product data. The content can be normalized and stored into a hyper-graph structure in the servers. The system can be used to select a portion of the hyper-graph structure based on a particular customer and to generate a recommendation for the particular customer based on the content in that portion of the hyper-graph structure. The system can also generate personal catalogs based on the information in the hyper-graph structure. The system can perform competitive analysis between products from different sources and include the results in the recommendations. Moreover, the system can perform a vertical analysis of consumable products to provide recommendations for tools or products that can be used in connection with the consumable products.

BACKGROUND

Commercial systems used by retailers and others to perform predictiveanalytics of customers, products, competitors and other data typicallyapply techniques such as ontologies, profile classifications,traditional data mining, and simple sales relationships when performingthe analysis. These techniques, however, can be limited in their abilityto extract and understand complex behavioral and contextual patterns onan individual basis as well as at various groupings of customers andproducts when used to perform recommendations, personalization, and/orcontent understanding, for example.

Further limitations and disadvantages of conventional and traditionalapproaches will become apparent to one of skill in the art, throughcomparison of such systems with some aspects of the present disclosureas set forth in the remainder of the present application with referenceto the drawings.

SUMMARY OF THE DISCLOSURE

A method is provided in which, in a system having at least one serverthat is operable to handle all types of data such as products,customers, competitor comparisons and general misc content, can bereceived. The data can be stored into a suitable structure that is ableto dynamically connect at various levels of granularity andinterpretation, such as a hyper-graph structure in one or more of theservers. A portion of these storage structures can be selected based ona individual pieces of content, such as a customer or a product, and byany type of ad hoc or formal groupings, such as taxonomies, marketsegments, and latent analytical groups that can be used to generate arecommendation or prediction for any other piece of data (e.g.,customers, products, sets of customers or products, latent groups ofdata).

Another method is provided in which, in a system having at least oneserver that is operable to handle commercial data, content includingcustomer data, product data and other types of data can be received. Thecontent can be stored into a suitable structure, such as a hyper-graph,in one or more of servers. Any portion of the hyper-graph can beselected (a sub-graph) based on a particular customer, a particularproduct, or any definable combinations of data in the hyper-graph, andcontent in the selected portion can be used to generate a new, organizedstructure, such as a personal catalog of products and related contentfor a customer.

Another method is provided in which, in a system having at least oneserver that is operable to handle commercial data, content includingcustomer data, product data and other forms of data can be received. Thecontent can be stored into a suitable structure, such as a hyper-graph,in one or more of the servers. A portion of the hyper-graph can beselected (a sub-graph) based on a particular customer, a particularproduct, or any definable combinations of data in the hyper-graph, andcontent in the selection portion can be compared to other structures, orused to find similar sub-graphs within the hyper-graph. Product andother data from different commercial entities stored in one or more ofthe servers can be compared. Recommendations and predictions for othercontent (such as a customer, product, or sets of customers or products)can be generated based on content in the selected portion of thehyper-graph and the comparison.

These and other advantages, aspects and novel features of the presentdisclosure, as well as details of illustrated implementations thereof,will be more fully understood from the following description anddrawings.

BRIEF DESCRIPTION OF SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 is a diagram that illustrates an example of a commercialenvironment having a complex processing system using emergent data.

FIG. 2 is a diagram that illustrates an example of a distributedarchitecture for the emergent data system.

FIGS. 3A and 3B are each a diagram that illustrates an example of ahyper-graph that is used for handling content in the emergent datasystem.

FIG. 4 is a diagram that illustrates examples of functional modules inthe complex system.

FIG. 5 is a diagram that illustrates an example of a personalizationformula used by the complex systems and emergent data system.

FIG. 6 is a screenshot that illustrates an example of a personal cataloggenerated by the emergent data processing.

FIGS. 7A-7C are each a screenshot that illustrates additional examplesof personal catalogs generated by the emergent data processing.

FIG. 8 is a diagram that illustrates touch points and feedback in theemergent data processing system.

FIG. 9 is a diagram that illustrates an example of privacy policies inthe emergent data processing.

FIGS. 10A-10C are each a diagram that illustrates example of performancebenchmarks produced to evaluate the emergent data processing in makingpredictions and recommendations.

FIGS. 11A and 11B are each a diagram that illustrates additionalexamples of performance benchmarks produced to evaluate the emergentdata processing in making predictions and recommendations.

FIGS. 12-14 are each a flow diagram that illustrates examples of stepsfor data handling by the complex systems and emergent data processing.

DETAILED DESCRIPTION

Certain implementations described in the disclosure relate to a methodand system for emergent data processing using complex systems. Whencompared to other systems, an emergent data processing complex systemcan provide features that those other systems may not be capable ofsupporting. The emergent data processing system can providerecommendations and predictions through various means, but not limitedto, online, mobile, email, in-store, catalogs, circulars, and directmail. Moreover, the emergent data processing system can support featuressuch as services, promotions, clubs, agents, dynamic pricing, trending,sentiment, internet sources, quality assessments, and user interfacedesigns. Other features supported include customer satisfaction (CSAT),internal communications, verticals, loss prevention, consumables, parts,activities, subjects, data integration, predictive intentions, time andlocation, merchant control, marketing control, and finance control. Inaddition, the emergent data processing system can support offers, deals,advertising, behavior, context, granular crowds, latent networks, socialnetworks, apparel merchandising, appliances parts and services, and factnavigation.

The emergent data processing system, which can also be referred to asthe emergent catalog or TEC, uses complex systems to handle emergentdata and its processing. Emergent data can refer to data, such asproduct or customer data, which can dynamically change, evolve, besummarized, re-combined, modified, and/or be adjusted as a result ofadditional information being considered and/or in response to analysisperformed on the data. With respect to the handling of recommendations,predictions, personalization, behavior and content understanding, forexample, the emergent data processing system can be based on the notionthat the data and its interpretation and use are very complex, elusiveto traditional techniques that attempt to interpret a human's point ofview or need, and translate this into execution-able data and processes.Accordingly, the emergent data processing system can normalize incomingdata into a hyper-graph type (or similar) structure that is sufficientto contain these complex relationships, and can then use variousartificial intelligence (AI) and/or reasoning techniques applied toreal-word use-cases (e.g., human descriptions and needs) to extract andunderstand patterns and data based on the graph's persona (e.g.,characteristics, interconnections, patterns and interpretations).

The emergent data processing system can be used to provide a deepunderstanding of any piece of data in the graph, such as a customer, acompany's content (e.g., products, offers, deals, supportingcollateral), and a competitor's related content. With thisunderstanding, the emergent data processing system can enableapplication developers to draw out ideas, patterns, predictions, and/oranswers to various issues. Application developers are routinely asked toadd functionality to retail sites, for example, that would includevarious kinds of product recommendations on product detail pages, or toremind customers of items they may have forgotten based on what theyhave in their cart during checkout, or for recommendations that are inthe area that a mobile phone user is currently walking through. Inaddition to recommendations, there are many applications wheredevelopers lack the subject-level skills but have the technical skillsto augment assortment management systems, pricing comparison systems,loss prevention and fraud control systems, and numerous forms of contentmanagement especially in the area of large marketplaces. All of theseareas can be easily accessed in the emergent data processing system viaits API calls. For any particular customer, for example, moreappropriate decisions can be made and/or more appropriate options can beprovided based on the context, behavior, events, interests, preferences,and/or activities of the customer. These results can be achieved whileconcurrently keeping these ideas and predictions relative to the needsand directions of a company's financial goals, marketing andmerchandising requirements, and then ensuring that the results arepositioned in a favorable way against the company's competitors. Theserelationships are implemented without attempting to fashion direct linksand rules that are not readily possible because of the inherentcomplexity.

FIG. 1 is a diagram that illustrates an example of a commercialenvironment having an emergent data processing system. Referring to FIG.1, there is shown a commercial or retail environment having an emergentdata processing system 100, which can also be referred to as theemergent catalog system or TEC, for example. The environment can alsoinclude a network 110, users or customers 130, competitors 140,third-parties 150, company information 120, and products/itemsinformation 130.

The network 110 enables the various components in the environment tocommunicate with each other. In one implementation, the companyinformation 120, which can include business or marketing information,for example, can communicate with the emergent data processing system100 directly and/or through the network 110. Similarly, theproducts/items information 130, which can include a wide range ofproduct information for a particular retailer or commercial entity, forexample, can communicate with the emergent data processing system 100directly and/or through the network 110.

Users 130 can provide information to the emergent data processing system100 and/or access information from the emergent data processing system100 through the network 110. Similarly, the emergent data processingsystem 100 can obtain information from competitors 140 and/orcommunicate with third-parties 150 through the network 110.

Because the emergent data processing system 100 uses information fromusers, customers, competitors and other data types to perform itsfunctions, more detail information for some major data types is providedbelow in this regard.

Customers

A customer (or other types of data that is traditionally associated withprofiles) in the emergent data processing system 100 does not have astatic profile. Instead, a customer can be described as a dynamiccollection of interconnecting dimensions of data, where each customercan be defined by not only characteristics about that customer, but canalso be defined by relative data from other customers. In someinstances, a customer can be defined by what he or she is not. Acustomer in the emergent data processing system 100 can more correctlybe characterized by its collection of associated data, such as interests(specifically identified by the customer or inferentially determined),probable activities the customer is engaged in (inferentiallydetermined), specific attributes about a customer, such as maritalstatus, age, number of kids, where they live, all of which can beidentified specifically by the customer or inferentially. Each customeralso can have a detailed set of behavioral and transactional dataassociated with them that defines things they do and when. The emergentdata processing system 100 is then able to compare all of the customersand find those whose data largely intersects. From this, customers withintersecting data can copy data from one another, filling in missingpieces. This process can extend in more than one dimension, intersectingand copying data from multiple sets of customer-computed segments,weighting the probability accordingly as missing pieces are filled in.In this way, it can be seen that every possible aspect that the emergentdata processing system 100 captures and measures can be related to everycustomer, with some weight scoring how much of each element is likelytrue for each customer. This can continue to improve for all customersas every new piece of information comes in about any individualcustomers. It also can be used to determine, based on a customer havinga low weight in some elements, what characteristics a customer likelydoes not have. In this way, a customer has every possible ‘persona’ ordifferent collection of characteristics. Which persona is active anddominating any interaction is what the emergent data processing system100 must determine at runtime when the interactions occur.

A customer can be dynamically defined, and the various components orparts that define the customer can be distributed through thehyper-graph type structure as either positive graph points or negativegraph points. As such, the emergent data processing system 100 need notuse a typical notion of a customer profile. For example, the concept ofa customers' profile, when looking to discover the deeper levels ofcontext, behavior, events, characteristics, and/or other features of thecustomer, becomes valid when interpreted as part of all customers. Thus,the emergent data processing system 100 can be said to have onecustomer, but that customer has an infinite number of variationsdepending on the exact context of data at any instant in time.

In the emergent data processing system 100, a customer can be largerthan a single unit that is typically thought of as a customer. Acustomer in the emergent data processing system 100 can have a verylarge number of personas, each of which can be captured from fragmentsof an individual context, and then collected together based on sharedcontexts. As a result, every customer can be the collection of allcustomers at a point in time for a given context, with thecharacteristics and predictive behavior extended from all theintersections, not just the individual. Any one instance of a customercan represent proxies of particular cohorts, segments, micro-segments,and collections. A customer in the emergent data processing system 100can represent an entity for which certain missing information can befilled in or patched from the contextual connections to other customers(e.g., contextual parents).

From the description provided above for a customer in the emergent dataprocessing system 100, the complex characteristics that become availablefrom the dynamic data dimensions, the relative interconnections of thecustomer dimensions (e.g., what is and what is not the definition ofeach customer), the large number of semantics variations (e.g., context,behavior, events), and the slices of persona that each customer is madeup of, which creates a view of a customer that is otherwise not able tobe captured through the typical profile and ontology type approaches.The definition of the customer can be based on the rich expressions ofwhat the customer is doing and how these events intersect with all othercustomers. The emergent data processing system 100 provides the abilityto capture this type of data, and then the ability to use it in manytypes of applications. An example would be where there are 50 millioncustomers, and they have resolved to a set of 1 million differentpersonas (abstracted collections of context, behavior, interests,activities, and/or other criteria). When interacting with a customer,any one of the 1 million personas is available to help determine whatthe customer is likely to be doing next (for predictions andrecommendations, for example). Clues from real-time searches,navigation, time, location and weather, for example, can be used todetermine which of the personas is most like prevailing, and from thatidentification, any upcoming recommendations and predictions can beslanted towards that prevailing persona.

Competitors

Each piece of content that is loaded or stored into the emergent dataprocessing system 100 can be automatically connected in many dimensionsto other content. In some instances, the content from one content sourcecan be connected and can compare itself (e.g., its sub-graph in thehyper-graph) to content from another data source (of similar ordissimilar content). The content can represent products from oneretailer connecting and comparing themselves to the same or similarproducts from one or more additional retailers. Any source or content ofany source can be connected and compared to any other source or contentof any other source. Comparisons are made using all of the dataavailable from both the competitor and the source company. This willinclude specific data carried in a product, for example, like title,price, brand, color, description, numerical and technical definitions,packaging, and also include external data that relates to any of theitems being compared, such as blogs, reviews, ratings and taggedpictures. In addition, data that is related to the understanding of thecontent is also used. For example, one product may use a term or anacronym, while the next product uses descriptions of these terms oracronyms. The emergent data processing system 100 can use data that ithas learned through associations to these terms, to expend and comparethem as they are encountered. In addition, when items are determined tobe the same, the characteristics that have been used uniquely from oneare then transcribed to the other, making a common definition that canthen be used to better understand a third.

When connecting one piece of data to another piece of data, the emergentdata processing system 100 can create a type of dynamic data-mart usingmany dimensions of data as selection criteria. A data-mart may refer toa subset of the data that is being warehoused (a sub-graph) or stored,or to an access layer (e.g., application programming interface or API)that can be used to access the data that is being warehoused or stored.Using the data-mart it may be possible to initially create a pool ofitems that are likely to connect to the input item. The particular termsfrom the input item and those of the targeted items need not be reliedon in every instance because certain items can be expressed using uniqueterms. By using many different selection dimensions, the emergent dataprocessing system 100 can substantially guarantee that the items thatare to be compared are the ones that are most needed for the comparison.That is, the emergent data processing system 100 may not be affected, asother systems can be, by performance and/or scalability issues becausethe emergent data processing system 100 need not compare every item toevery other item, particularly when in many instances the connectionsthat would be compared tend to provide little to no real value. Forexample, as an initial selection that would reduce a potential list ofcompetitor items from tens or hundreds of millions of possibilities, onecould select (in parallel and results combined) based on brand,technical terms, numbers of a certain size, attribute characteristics,composites of title or description matches at a theme or n-gram level,model number, fuzzy interpretations of model numbers (one has a specialcharacter and the other does not, but could still match), genderreferences, color references, discussion of packaging or refurbishing,bundle identification, and/or accessory identification. These can bedone individually and in combination. The results should be a moreworkable list size of around 50 thousand items, which can then easily beprocessed in parallel and ranked.

Once the list of comparable items has been selected, the emergent dataprocessing system 100 can compare each item in the target to the inputitem, in many different dimensions. The comparison can include thecomputation of a score that represents the degree of overall comparison,as well as individual numbers that represent the degree of comparisonalong different lines. For example, a comparison line can be howcomparable two items are when only technical terminology is used for thecomparison.

The emergent data processing system 100 can also compute a largeconnection of sub-graphs that compare to other sub-graphs, and canprovide many metrics that show the semantics of how the sub-graphscompare. This comparison would evaluate both the intersection of oneartifact's detailed sub-graph against all others that it connects to,and also compare one artifact's sub-graph to a sub-graph of associationsthat extend each node of the artifact's sub-graph, and then compare theassociations against any target sub-graphs. Thus, an association ‘cloud’can be placed in the middle of two sub-graphs, which enable a comparisonthat is not determined by the exact words or themes involved in eitherof the two individual sub-graphs. This feature of the emergent dataprocessing system 100 can be useful because for many applications one ofthe requirements can be to provide a deep understanding of how similarany two items are, where they are similar, and where they aredissimilar.

While the emergent data processing system 100 creates its graphconnections, it can create semantic dimensions that measure theassortments of items, that is, those with specific levels of similarityacross sources (meaning comparing any collection of items to any and allother collections of items in the graph). The evaluations made can beused to determine characteristics of a source's items, such as pricepoints, measurements of relative content quality or lack of contentquality, assortment coverage, and characteristics or attributes aboutcollections of items to, for example, determine the likelihood of itemsmissing certain characteristics or attributes.

The hyper-graph generated by the emergent data processing system 100 canbe used to produce price recommendations that can save considerable timeand effort in projects that require dynamic pricing based on thecompetition. Current programs work based on the principle that thesource company can provide all the data points and algorithms to bestcompute what the price should be. The emergent data processing system100 can perform a much simpler computation based on the ability of thesystem to very accurately measure the similarity of items to build acomparable assortment—by identifying comparable items in thecompetitors' information, and then using their prices to determine whatthe best price points should be. This approach then lets competitorsspend their resources (e.g., money, time) on processing and computingthe low-level metrics, while the emergent data processing system 100 canuse the pricing information from the competitors to determine its ownprice recommendation points.

FIG. 2 is a diagram that illustrates an example of a distributedarchitecture for the emergent data processing system. Referring to FIG.2, there is shown an implementation of the emergent data processingsystem 100 of FIG. 1 that is based on a distributed architecture havingmultiple servers which are able to process requests in parallel. In theexample shown in FIG. 2, the emergent data processing system 100 caninclude servers 200 a, 200 b, . . . , 200 n.

Each of the servers can have a processor 220 that is operable to performcomputations, instructions, processes, and/or methods related to thehandling of emergent retail or commercial data. The functions performedby the processor 220 can be based on one or more applications that canbe executed on a central processing unit (CPU) or similar integratedcircuit in the processor 220. The CPU (not shown) can run an operatingsystem on which the one or more applications can be executed.

Each of the servers can also have a memory 230 that is operable to storedata and instructions related to the handling of emergent retail orcommercial data. The data can include product data, customer data,business data, and/or other type of data received by the emergent dataprocessing system 100. The instructions can include instructionscorresponding to software applications or other operations performed bythe CPU in the processor 220.

Additional examples of features and/or operations that can beimplemented as part of an architecture of the emergent data processingsystem 100 are provided below.

Architecture

Distributed Semantics

As described above, the emergent data processing system 100 can beimplemented as a highly-distributed, parallel processing system. Forexample, the emergent data processing system 100 can be implementedphysically across many servers and/or storage devices. The storagedevices can be part of the servers or can be separate from the servers.Moreover, the emergent data processing system 100 can behighly-distributed semantically such that data can be connected,distributed, summarized, normalized, and/or otherwise handled, acrossmany different dimensions, without the notion of database normalization.

Data Integration—The emergent data processing system 100 can normalizeincoming data and can connect the incoming data across many dimensionsto other related data. The connection of incoming data to the other datacan be performed automatically. Incoming data would be from all sourcesof related data within the company, such as transactional data fromdifferent retail areas, data from content management systems or customerservice systems, and data from customer maintained profiles. Data canalso come from external sources that would be able to be linked tointernal data, such as other information about a customer that can befound on the internet and linked based on street address, phone number,and email address. Internet data can also link product data fromexternal reviews, blogs and other sources based on model or universalproduct code (UPC) numbers, as well as by using the emergent dataprocess system 100 capability to perform ‘fuzzy’ matching of content.External data can be provided by various license and access methodscommon to the internet. The emergent data processing system 100 canhandle dirty data, that is, data that is incomplete, not wellunderstood, too difficult to connect to other data, out of date with itsdocumentation, and/or changing often. The dirty data can be used by theemergent data processing system 100 to provide solutions and generateresults while other system may simply disregard such data. The emergentdata processing system can also fill-in missing data based on sub-graphoverlays from similar items.

Data Unification—The emergent data processing system 100 can break downthe data that is integrated into very low levels and can interconnectthe broken-down data in the hyper-graph. By this approach, the data canbe considered to be the data itself and need not be defined by externalsemantics. The emergent data processing system 100 can enable the datato be unified into patterns that can be compared, contrasted andotherwise handled in order to understand the relative significance ofany piece of data.

Content Delivery—The emergent data processing system 100 can be operableto break down, normalize, unify, and self-define the data. Nevertheless,the emergent data processing system 100 can also be capable ofreconstructing the data in more traditional formats and of deliveringthe data with new meanings and/or tags to systems that are not suitableto deal with the type of data representations supported by the emergentdata processing system (e.g., fuzzy representations). For example,content management system may want to have tags on data that is poorlywritten, missing important attribute definitions, classified in thewrong taxonomies, or otherwise of note. The emergent data processingsystem 100 can make these assessments and then update the appropriateareas in the content management system with its determinations, orinform the responsible personnel of changes that it recommends. This canthen let the content management system operate in their traditionalforms, as opposed to using more complex, fuzzy interpretations of thedata via graphs.

Social Networks—The emergent data processing system 100 can representdifferent types of semantic structures in its hyper-graph, not only ofan individual item, but of connections such as social networks. Socialnetworks represent connections between people—friends, experts in asubject, items that people are following, individual recommendations forparticular products . . . . The network as a whole then can come intoplay when considering applications such as recommendations, bytraversing the networks and measuring semantics and distances from startpoints. For example, after create a set of recommendations for aparticular user based on traditional recommendation techniques (e.g.,prior sales), the emergent data processing system 100 can evaluate allof the recommendations against information present in the individual'ssocial network, which may intersect in various ways to the items beingrecommended. These intersections can then cause an individualrecommendation to be increased or decreased in weight. The emergent dataprocessing system 100 can also create its own latent networks, forexample of experts in any subject area, groups and their members, socialInfluencers, users who show followings that depict trust or strongreputations. As an example, a gardening group could list its members,which all possess information about their interests, purchases andactivities. This information can then be used to influencerecommendations that someone outside the gardening group may need. Inthe same way, it can be determined who the top people are showing themost interest in gardening, and this can then form a virtual or latentgroup of people that function in the same way as a specific group would.These networks can be constructed dynamically and can be used, just likemanually created social networks, to augment applications or providespecific recommendations based on their characteristics.

Fraud Detection and Loss Prevention—The emergent data processing system100, because of its ability to integrate disparate, incomplete andotherwise noisy data, can perform multiple functions that enable it toprovide utility for loss prevention and fraud detection. For example,the emergent data processing system 100 can identify customers whoseoverall activity across a wide spectrum of enterprise touch points wouldsuggest common or uncommon behavior. In another example, the emergentdata processing system 100 can integrate many different pieces ofcorporate data (including dirty data) to generate and provide a unifiedview of the customer that otherwise is not obvious. In these instances,the emergent data processing system 100 can not only base its analysison traditional data such as sales, but can also include the way thecompany has been interacting with the customer and how this interactioncan be interpreted relative to fraud. Different fraud types wouldinclude how layaway purchases are made and paid for, how collusionbetween marketplace sellers and fraud purchases are identified, how tobetter determine that a customer is identifiable as a good customerthrough the integration of more data that relates to that customer butmay be outside of the immediate retail need

Distributed Customer Data

The emergent data processing system 100 can connect data in multipledimensions and can let an application suggest the data that must bedrawn upon and how that data would be dynamically connected andinterpreted. This is generally the case when complex use-cases are beingimplemented, and it is not apparent in advance about how the desiredoutcome is going to be determined. The application architecture is suchthat it can review many result sets from existing APIs, based on theinput being used by the use case, and determine which APIs are gettingthe appropriate data returned or as close as possible. Then differentmanipulations of the data in the API can be used that can produce a morenarrow or a more generalized set of results, or cause other variationsbased on slight input alterations. An aspect is then to narrow down theneeded results by expanding and filtering as needed against the resultsof somewhat related APIs.

Characteristics Interpretation—The emergent data processing system 100can define personal characteristics and then represent those personalcharacteristics as a sub-graph or set of sub-graphs in the hyper-graphstructure. Personal characteristics include such data points asinterests, hobbies, activities, behavioral trends, and relationships toother people based on these characteristics. The emergent dataprocessing system 100 can be capable of fine-tuning definitions thatintegrate not only the simple surface-level data such as what issupplied in a profile question, but also the use of context and behaviorto re-define or augment the characteristics.

Reasoning—The emergent data processing system 100 need not use areasoning technique because the system itself is a reasoning system thatis using the hyper-graph to represent different use-cases, contexts,characteristics, and other features that are used by the emergent dataprocessing system 100 to analyze the data and make decisions, includingdecisions regarding recommendations, the interpretation of content, anthe identification of market segments, to name a few.

Predictions—The emergent data processing system 100 can performpredictions by using a new context (e.g., sub-graph) to relate otheritems against. The emergent data processing system 100 can rate itemsagainst different contexts, can predict the likelihood of the item beingpart of the targeted context, and can produce a score for theprediction. The rating can be performed automatically for an item andthe item can be rated against multiple contexts.

Context—In predictions, the definition of a context (i.e. weather,location, time, event restrictions) can cause the artifacts of theemergent data processing system 100 (e.g., customers, products,patterns) to be processed within that context.

Behavior—The emergent data processing system 100 can handle differenttypes of behaviors. A behavior is a time-oriented set of events that anartifact (e.g., person, product, pattern, event) can participate in, howthose events are set as sub-graphs in the hyper-graph, and how themovement and intersection of these sub-graphs can occur with otherartifacts or sub-graphs. This flexibility enables the emergent dataprocessing system 100 to use a very rich interpretation of behavior,unlike simpler systems that may use click streams or other types oflogs.

Anonymity—The emergent data processing system 100 can break apart acustomer into observable micro-segments, where, when all possiblemicro-segment possibilities (for all customers) are determined, eachcustomer then contains each of these micro-segments (e.g., from 0 to 1).This means that every customer contains the characteristics of everymicro-segment (considering that a score of 0 still refers to thecustomer containing no part of that micro-segment). These micro-segmentscan then be applied to customers that are unknown (e.g., customers notlogged in) based on their behavior, context, and/or content, which couldbe expressed anonymously through their browse/search type activities.These activities can intersect with many micro-segments, whichcollectively can form together and be used by the emergent dataprocessing system to identify the most related customer with thosespecific micro-segments, and use that customer as a proxy for thecurrent action the customer is taking. These steps can continue overtime and the results can change in real-time as the emergent dataprocessing system 100 continues to learn more about the customer.

Cohorts—A cohort can correspond to one of the micro-segments of acustomer as described above.

Crowd-sourcing—The emergent data processing system 100 can build datathat is a summary above the individual customer. This type of data canbe referred to as crowd-sourced data, for example. It can be produced atthe cohort/micro-segment level. The emergent data processing system cancontinue to accumulate crowd-sourced data using combinations of cohorts,cohorts mixed with other data types such as networks, networks on theirown. The crowd-sourced data can be a summary at a level above theseother types of data. The crowd-sourced data can be used by the emergentdata processing system 100 to perform prediction routines, where thelevel of inclusion of any customer into a crowd-sourced data definitioncan carry forth the characteristics of the other customers thatintersected that same crowd-sourced data.

Segmentation and Micro-Segmentation—As described above, the emergentdata processing system 100 can perform segmentation andmicro-segmentation in connection with anonymity, cohorts, and/orcrowd-sourcing, for example.

Latent Networks—As described above, the emergent data processing system100 can perform various functions and/or operations for latent networkssuch as social networks, for example.

Targeting—The emergent data processing system 100 can perform functionsto target customers based on a very detailed, bottom-up understanding ofmany aspects of the customer, and how they relate to the needs of thebusiness as expressed in offers, deals, ads, and the like. The emergentdata processing system 100 can use the instructions or guidance providedby a business (e.g., guidance of how often to contact a customer, theimportance of acquiring a new customer in view of revenue or margin) andcan make determinations of what is the best marketing campaign given thecurrent context should be.

Events—In the emergent data processing system 100, events can refer toany activity that happens in the system that can be related to a time.The emergent data processing system 100 can take the events and thenrelate and intersect them throughout the hyper-graph, which can bringthe information and content in the graph into any interpretation of anyevent. The emergent data processing system 100 can fill most missinginformation in an event by using probability measurements—findingsimilar event sections in other events that match a context andmicro-segments, and fill in the missing data from that similar eventdata.

Triggers—The emergent data processing system 100 can be operable toenable triggers. Triggers can refer to the ability for marketing,merchandising, finance, or some other organization within a company touse the emergent data processing system 100 to create the expectation ofan event and a context, and what to do when the system sees a customerwhose actions match the trigger. This allows for a more one-on-one typecommunication with the customer (or automated connections ofcommunications-like behavior between any two pieces of data), as thespecifics of who the customer is, what the customer is doing, and/or thecurrent context surrounding the actions, can be defined withspecificity, which allows for messaging that is generated by theemergent data processing system 100 in connection with a trigger to alsobe specific. For example, a trigger could be set so that if any querythat a customer performs contains fishing poles in the result set, andif a specified time of year, location and/or weather pattern is alsopresent, then the system could return data based on ice fishing.

Actions—The emergent data processing system 100 can be operable toenable actions, which are similar to events. But unlike an event, whichcan refer to the recording of the customer's clicks and other behaviorin the system, actions can be designed to be expressions (e.g., twitter)that state something external that a user is doing, which leads to moreconnections in the hype-graph, which can later have an effect onrecommendations made in connection with a customer. So if a customertweets something about going on a vacation to Mexico, any subsequentinteractions with that customer in the retail system could be weightedtowards Mexico and vacations, without the customer specifically askingfor that.

Subjects, Topics, Themes and Activities—In the emergent data processingsystem 100, each event or action of a customer can be part of acollection of sub-graphs that are used by the system to predict theintended activity behind the event. These activities can be summarizedto determine the probable intended subject area. For example, variousevents around telescopes may yield an activity around how to build yourown telescope, which could then connect to subjects such as astronomyand mirror polishing. Lower levels of the activities can also beexpressed in topics and themes, which are progressively more detailedbreakdowns of the activity. These more detailed breakdowns of theactivity can then be part of the connection to latent networks, whereexperts and other designations of users can be computed for differentactivities, subjects, themes and topics. These computations can be builtby specifying a context, using various algorithms to determine the setof customers that are a best match to that context, and then determiningwhat subjects, topics, themes and activities those customers are alsointerested in.

Distributed and Dynamic Content

The emergent data processing system 100 can be operable to compute,assemble, disassemble, and/or generate content in multiple forms. Beloware described some examples of the content handling features that can besupported by the emergent data processing system 100.

Personal Catalog—The emergent data processing system 100 can generate apersonal catalog from the information and structure of the hyper-graph.The catalog can be a set of sub-graphs from the hyper-graph that areconnected together in a visual format. The sub-graphs can be of any datadimension, such as personal information about a customer, how thatinformation intersects with current ‘hot’ products, which of thoseproducts are best positioned from price against the competitors, andwhich of these products have not been purchased by the customer, andwhich similar customers have purchased these products at this time ofthe year. All of these functions are represented by sub-graphs that areoverlaid and a unified catalog created from them. Also, existing data,such as an email marketing campaign that is sent can be automaticallyturned by the emergent data processing system 100 into a personalcatalog and expanded or extended through various sub-graphs. In someinstances, a specific aspect of a customer, such as their interest inastronomy, can be formed into a personal catalog. Moreover, a largerportion of what is known about a customer can be formed into anexpansive ‘master’ catalog for that customer. In addition, the catalogsthat are generated for a customer can be connected together into acatalog collection, making it easy for the customer to find individualcatalogs by type or date. This connection of catalogs can be madeautomatically by the emergent data processing system 100. Examples ofpersonal catalogs are described below with respect to FIGS. 6-7C.

Opportunities—Opportunities are specific types of catalogs that can begenerated by the emergent data processing system 100. These types ofcatalogs can be produced based on context, behavior, message, and/oroffers. Context can refer to the conditions that may be needed for thecatalog to be activated. Behavior can refer to the actions or events thecustomer may need to do for the catalog to be activated. Message canrefer to what a marketing organization wants to specifically say to thecustomer in the given context and behavior. The message information canbe provided to the emergent data processing system 100 by the marketingorganization for handling. Offers can refer to that which the customeris to obtain or receive once the context and behavior are valid and themessage delivered referencing that offer. The emergent data processingsystem 100 can perform the validation of the context and/or of thebehavior. For example, the emergent data processing system 100 can checkthat the necessary conditions, actions, or events have been met toactivate the context and/or the behavior.

Dynamic Emails—The emergent data processing system 100 can provide areal-time method for asking for various types of recommendations,delivered when the customer actually opens the email (instead ofactually computing the email content and then sending it, dynamic emailssend the email header, and the body is computed in real-time only whenthe customer opens the email. In this manner, the emergent dataprocessing system 100 can enable marketing to move much quicker indesigns and eliminates many steps that it normally takes if a variableemail is to be pre-computed. Examples of improvements in the steps:

Creating custom images

Manually loading images into the email

Manually configuring text

Approving each offer and the creative for each offer for every email

Huge savings when multiple changes were made to offers often requiringreformat of the offers each time.

Localization—The emergent data processing system 100 can generaterecommendations that are based on regions, areas, and/or closest stores,for example. These regional or location-based recommendations can bepre-defined (e.g., a particular store) and/or can be created based on aparticular campaign (e.g., define an area that is specific to thecampaign).

Bundles—The emergent data processing system 100 can compute generic andpersonalized bundles of recommendations in many different formats.Examples of formats include, but need not be limited to similar items,bundled together purchases, bundles based on subjects, topics, and/oractivities, and bundles based on expected future purchases targeted tospecific dates. The emergent data processing system 100 can compute thebundles based on current needs or based on predicted needs at aparticular point in time.

Search—The emergent data processing system 100 can support numerousmethods (e.g., algorithms) for enhancing search results. These methodscan range from crowd-sourcing to being oriented around a subject oractivity, or very personalized to the individual. This allows for theconstruction of customized searching in many different ways. Forexample, a traditional search for polo shirts may list the shirts basedon general purchase statistics. An enhanced version of that search couldfactor in the current customers favorite color or brand, causing thoseitems to rank higher.

Browse—The emergent data processing system 100 can enhance the resultsfrom browsing in a similar manner as with the searches described above.In a typical retail web site, products are often listed under ataxonomy, allowing the customer to browse from the top to the bottom byselecting an item from each level that refines the set of possibleproducts. As in search, the listing of the taxonomy elements can be madein an order that is more probably of interest to the individualcustomer, and the product results can be sorted based on personalinformation, such as color and brand preferences.

Accessories, Add-ons, Interests, Ideas—The emergent data processingsystem 100 can support and perform different types of analysis and/orinterpretations of any graph point in the hyper-graph. For example, fromthe point of an individual product, the emergent data processing system100 can determine or identify accessories, add-on products, replacementor upgrade products, interesting product ideas based on the product, ornew ideas for products based on that product, to name a few. So if thecustomer purchased a television, the emergent data processing system 100could display a set of accessory items for the television, such as aDVR, cables and a wall mount. A combination of sales information, price,and content definition would be used to determine the set ofaccessories.

Promotional Interaction—The emergent data processing system 100 can beoperable to evaluate different types of activity between a customer anda company. These evaluations can be based on other personal and cohortinformation. Once an evaluation is performed, the emergent dataprocessing system 100 can determine the next communication step betweenthe company and the customer. This approach differs from traditionalpromotional campaigns that work by primarily creating the campaign andthen trying to determine which customers best fit the campaign. Thecustomer-centric method supported by the emergent data processing system100 can determine promotional information that is best suited for aparticular customer by providing meaningful opportunities for thatcustomer as opposed to just offering any type of discount to thecustomer on products or items that may be of limited interest to thatcustomer.

Associations—The emergent data processing system 100 can continuouslyand/or repeatedly learn how to understand and interpret the data in itshyper-graph. An approach to enable such understanding is through thecomputation and use of associations. Associations can refer tosub-graphs that can be dynamically formed around an activity, a subject,a person, a product, a topic, a word, or the like. These sub-graphs canthen be compared, contrasted and interpreted relative to its place inthe hyper-graph overall, and provide the intelligence that the emergentdata processing system 100 uses to make its decisions.

Recommendation Formula—The emergent data processing system 100 canimplement, perform, and utilize a personalized formula for its decisions(e.g., recommendations). The formula can apply to different types ofcontent and not just products. An example of a personalized formula isdescribed below with respect to FIG. 5. There are two generalcharacteristics of the data that can be used with the formula to supporta large number of different types of recommendations: similarity andsemantic connections.

Similarity can refer to the ability of the emergent data processingsystem 100 to measure the similarity or likeness between any two itemsas a relevant part to its decision making. Using this technique, anartifact of any type in the emergent data processing system 100 can beconnected and rated against any other artifact. Unlike other systemsthat work by matching specific pieces of information such as a UPC codeor a manufacturer model number, the emergent data processing system 100works be reading, evaluating and comparing the detail data used indescribing any piece of content. In addition, different sources of thesame content often describe the item differently, and the emergent dataprocessing system 100 can link these together and use their collectivedata as a definition. Many different dimensions of the data are computedand their sub-graphs compared. This would include the theme of the data,its technical level of wording, use of numbers and proper names,specific items such as color, brand, and if the item is refurbished, andother similar measurements. Each measurement consists of a scoremeasuring its total amount of data of that type, and then another scorethat rates the intersection of both artifacts for each element measured.

Unlike similarity, which can be computed through the characteristics ofany content item, semantic connections can refer to connections betweentwo items that are not based on similarity. For example, a televisionthat is connected to a television stand or to a video cable. Theconnection in this example is not one of similarity but one of use. Theemergent data processing system 100 can be operable to look for and useany type of connection between data items, which enables the system toprovide a richer understanding of the items.

Feedback And Learning—The emergent data processing system 100 canreceive, produce, and analyze feedback. For example, the emergent dataprocessing system 100 can learn whenever it makes some type of decisionor recommendation that can result in a user providing feedback. Anexample of this type of learning can occur when a recommendation is sentas an email to a customer, who in turn opens the email and clicks on aproduct shown in the email, reinforcing the connection of product tocustomer and cohort in the given context of how the email was sent. Theemergent data processing system 100 can analyze this response and raisethe importance of similar recommendations to other customers. When thecustomer opens an email and doesn't click on any of the products shown,the emergent data processing system 100 can lower the importance ofthose products to similar customers. An example of the use of feedbackin the emergent data processing system 100 is described below withrespect to FIG. 8.

Finance, Marketing, and Merchandising Integration—Different parts of thebusiness can introduce strategic goals into the emergent data processingsystem 100. Those goals can play a part in the decision making of theemergent data processing system 100 (e.g., recommendations). Forexample, finance could indicate that they are interested in margin, ormaybe new customer acquisition. Marketing could indicate that they areinterested in getting people to open emails better. Merchandising couldindicate that they are interested in promoting apparel wheneverpossible. These kinds of goals can be set into the emergent dataprocessing system 100 and can be used to affect the decisions made. Theemergent data processing system 100 need not blindly follow these goals,but can take situations when the decision point can easily go inmultiple directions and use these goals to arbitrate the selection.Goals can be set at many levels of granularity, like specific to anindividual item in a specific context (such as an area of the country,time of year, type of weather), or very generally defined abstractions.Goals can use context, behavior (what actions or events should be takingplace), corporate information (margins, shipping costs, various sales,deals, campaigns in play) and any content (i.e. product collateral) todescribe the characteristics of the goal and how, when and where itshould execute. Goals can also be expressed both positively (must havesome characteristic) and negatively (should not have somecharacteristic).

Assortment Management—The emergent data processing system 100 cananalyze and review a company's products relative to a competitor andtheir similar products, comparing individual pricing, quality of thecontent written about the products, and overall assortment advantagesand disadvantages. The emergent data processing system 100 can performthe analysis and review automatically. Moreover, the emergent dataprocessing system 100 can use this information when makingrecommendations and other decisions in order to position the company'sofferings in a favorable light to the customer and the company.

Backtracking of Context—The emergent data processing system 100 cananalyze a user event (e.g., clicking on a category, doing a search) andgo backwards in time to a previous event to record that this was thenext sequential event. The emergent data processing system 100 canrecord the event back all the way to a new search, where the search hadno thematic connection to the previous search or browse. The emergentdata processing system 100 can indicate the probability of a new contextstarting. This can produce a path that shows the probabilities of theuser progressing in a particular direction based on the user's contextand cohort or crowdsourced-related contexts. The paths can beaccumulated individually based on context, which includes thematicdefinitions of the user as well as the current general context ofweather, location, and/or time, for example. In addition to browser andsearch type events, financial tracking can also used, recording orderamounts when an order is made. This can result in a higher follow-ratethan other events, since the path is one that eventually resulted in asale. So for example, a customer does various search and browseactivities while on a site shopping, and some eventually end if apurchase. When the purchase is made, the trail of activities that thecustomer went through can be recorded. Then, when new customers show asimilar type of activity, they can be guided in the results based on thegreatest likelihood to reach the purchase point, based on what thesuccessful purchase did.

Reasoning Process—The emergent data processing system 100 can performdifferent types of processes or techniques in connection with datahandling. For example, the emergent data processing system 100 can usequery agents, complex system type agents (discover top catalogs akaDTC), emergent result agents (data expansion catalogs aka DEC), bundlecreation and selection, predictions, targeting, and patterns. Thedifferent agents allow for many types of processing to be performed ondata, recursively, and eventually emerging with a set of results. Theresults are then normalized for consistent and relative-correctedoutput.

The emergent data processing system 100 can implement and use a complexsystem management system, which in turn can implement one or more agentsor independent processing techniques. Agents are typically given generaldirectives but not specific instructions. The results of functionsperformed by the agents are eventually normalized and combined, and anyissues can be patched in the agents by abstracting and generalizing thespecific issue. The process can use feedback processing because someresults can be circular. For example, one agent could be build that doesnothing more than try to locate any items that have the similar brand asan input item, but are more expensive. This agent could call other APIs(agents) that return results, and filter out any that do not meet itsrequirements. In this way, as new agents are built, they continue tofind new ways of locating the requested items for this agent,

The DTC can be a complex system agent that is widely used in theemergent data processing system 100. The DTC can use the results ofother agents (e.g., agents that are primarily query-oriented) todetermine the best or most appropriate catalogs (collections ofartifacts) with which they intersect. This can be a multi-level (e.g.,3-level) recursive process that can result in a set of catalogs andwhatever artifacts are contained under them, where the best catalogsrepresenting the emergent value of the agents are returned.

The DEC can take the catalog results from the DTC and can expand them(recursively if needed) and can intersect the results, proving aselection of artifacts that best represent the emergent nature of thecatalogs. These artifacts can be of any of the type of artifacts thatare typically placed inside the catalog (e.g., products, images, videos,news stories, reviews). These are then combined with the results fromthe other agents and can become the final set of results returned at themiddle processing agent layer.

Customers and products can come together based on bundle creation andselection supported by the emergent data processing system 100. The datain the emergent data processing system 100 that is related to a customercan be used to determine the areas of interest for a customer, then todetermine a selection of the main products, and finally to create abundle of associated products to the main products for a given contextor purpose. For example, a customer may have browsed on a product pageabout a particular polo shirt. From this, it could be determined thatthe customer has interests in polo shirts, of a particular brand andpotentially a particular size and color. Then bundles can be createdthat would first select a polo shirt, and then other products that goalong with polo shirts. The other products could also filter by color,brand and size if it applied. There can be many different bundle formulatypes, such as bundles that are all based on a particular interestshowing different alternatives, or bundles that focus on a specific itemand accessories that go along with that item, or bundles that focus on aspecific item and then other items that are eventually purchased over alonger period of time by other customers who both the focus item. Later,when a campaign or some process in the emergent data processing system100 needs recommendations for a customer, the select bundle process canbe called, which can evaluate the customer data and their bundles, andcan determine the best fit for the request.

The emergent data processing system 100 can use many different methods(e.g., algorithms) to assess customers, products, context, and behavior,for example, to make new assessments in the form of predictions, targetsand patterns. For predictions, the emergent data processing system 100can use cause/effect type algorithms to determine that based on somepast behavior a certain future behavior is expected (e.g., the purchaseof certain products based on prior purchases). For targeting, theemergent data processing system 100 can use a desired or preferred goal(i.e. marketing goals or finance goals) to compute the context andbehavior that would best yield that goal, delivering the context,behavior, products, logic and customers. Moreover, with respect topatterns, the emergent data processing system 100 can compute ageneralized view of probable purchases using various inferences againstcohorts (e.g., micro-segments).

Content

The emergent data processing system 100 can use different types ofcontent, from different types of sources. In many instances, theemergent data processing system 100 need not understand the data and howit could be used. The emergent data processing system 100 can handledata, even data that it may not fully understand, by expressing therelationships between data. For example, a field in the product datacould be called “code”. “Code” may not have any definitions, as theywere lost or poorly understood because of alterations over time.However, the understanding of Code can still be made based on the otherdata elements that it connects to—since all artifacts of the same codewould have a shared connection on code, and also possibly on otheraspects such as color or size. It could algorithmically be determinedthat code may function in some relationship in these items similar tosize, such as weight. APIs could be constructed that test this Code databecause it helps filter out other unwanted items. These relationshipscan correspond to data inside each individual piece of content as wellas how and where the content connects to other pieces of content. Evenwhen a particular piece of data is not fully understood or fullydefined, the emergent data processing system 100 can use of complexsystems and emergence to search out and take advantage of the existenceof any known data that can show a connection to something that is beingsearched for. This same approach can be used when considering negativepositions, that is, when something is being defined by what it is not.The more data that the emergent data processing system 100 is able toobtain, handle, and connect, the more the understanding of the data canincreasingly improve, which could provide more effective results inareas such as recommendations, personalization, and contentunderstanding. As an example, additional data can provide clarificationin determining what something is not. The inherent noise that exists indata (undocumented fields, data in documented fields that does not matchthe documentation, data that is understood but without any particulardirect need), particularly in ill-defined and not well understood data,may not be an issue in the emergent data processing system 100 becausesuch noise can be used as a source for determining what something isnot.

The emergent data processing system 100 can place data or content into agraph (e.g., a hyper-graph type structure) by working with the originaldata in its standard form as provided by its source. Examples of ahyper-graph and adding content to one are described below with respectto FIGS. 3A and 3B. The emergent data processing system 100 need nothave the data altered or modified into some type of format that may befound more useable because such alteration is often not required ordesired. For example, a product may have specific dimensions defined.These dimensions do not need to be made into specific database columns.They can just be stored in the graph with semantic tags that define whatis known about them. Graph traversals can determine what to do with thedata on a use-case basis. Moreover, when the data is altered or modifiedin some way, it typically means that some compromise in the data hastaken place. Instead, the emergent data processing system 100 can takethe original data and can determine how it can be placed into the graphwithout loss of complex characteristics that may be captured in thedata. Concurrently with the placement, the emergent data processingsystem 100 can normalize the data so that any application using the dataneed not have to deal with many pieces of data in many differentformats. The normalization allows the emergent data processing system100 to unify all the data types and sources, along with their complexexpressions of what they are and are not, into a single hyper-graphstructure that can then be easily and uniformly accessed.

The data that is handled by the emergent data processing system 100 canhave some ambiguity. The ambiguity can be semantic (e.g., like how theword ‘bank’ can be used in a particular piece of content). The ambiguitycan also concern factual understanding of the content. For example,there can be factual ambiguity when not everything about the content isexplicitly expressed but rather assumed to be understood. Both thesemantic and factual understanding of the content in the emergent dataprocessing system is part of the connecting and disambiguation processesthat can take place automatically on the integrated data, byinterconnecting the sub-graphs of every word, in its context of theoverall theme of the artifact, to the sub-graphs of other words in acontext, which makes the semantic connection and eliminates or reducesthe probability of ambiguity.

The emergent data processing system 100 can perform data disambiguationwith respect to at least four different areas: semantic similarity,semantic attributes, semantic relativity, and user interpretation. Withrespect to semantic similarity, a general set of associations of datacan be computed that show how one term's characteristics intersect withanother term's characteristics (e.g., how much of a synonym is one wordto another, based on a weighted system that show the number and strengthof connections as well as the absence of other expected connections).For semantic attributes, one or more terms can be considered to analyzeand determine the associations that best define how the term(s) arecharacterized (e.g., what are the characteristics of a lawnmower).

With respect to semantic relativity, the emergent data processing system100 can measure the relative positions in the hyper-graph structure of asub-graph that represents some object, such as a product, review, user .. . . An object can be any node in the graph. Moreover, from theperspective of the hyper-graph, an object can include any collection ofnodes and edges in the graph as well as being a single node in thegraph.

For user interpretation, the emergent data processing system 100 canperform semantic similarity, semantic attributes, and/or semanticrelativity but filtered specifically by what a user (e.g., any node inthe graph) is interacting with. This area can be considered as a levelof semantic understanding and disambiguation that is specific to anotherdimension like a user, instead of being relative to the semanticstructure of the content itself.

The content that enters the emergent data processing system 100 can bemeasured relative to other content that is classified by theircontributors, for example products submitted by a particular seller, andclassified under a particular taxonomy. Patterns of classification canbe computed, and outliers can then be assessed against their classes todetermine whether any item (e.g., product, review, image) is likely tohave been misclassified (e.g., inappropriate to a class) or ambiguouslyclassified (e.g., falls across a number of classes), the latter of whichcould indicate that there is a topological error in the actualclassifications or that an item that is too broadly defined. For eachitem, the emergent data processing system 100 can providerecommendations that suggest where it may be more appropriatelyclassified. One characteristic about this feature of the emergent dataprocessing system 100 is that it can determine the classifications basedon actual intersections of the items that make up each class and notagainst a definition of the class. Therefore, the emergent dataprocessing system 100 can determine the correct or most appropriateclassification of an item, or when there is ambiguity in the definitionof the classes for an item.

In addition to assessing the patterns of classifications or collectionsof items in the hyper-graph, the emergent data processing system 100 caninterconnect the patterns from different sources. This allows for anadditional dimension of data to show where items from one source (e.g.,EBay) can be connected with items from other sources (e.g., Amazon,YouTube). Any type of pattern representing any type of content can beinterconnected, as each pattern is itself a sub-graph, which is a node.And every node can be connected to any other node via an edge, and theedges are defined semantically, so any kind of connection between anytwo nodes is possible.

The emergent data processing system 100 can also operate as anintelligent delivery system for content. Once a set of data has beenloaded into the emergent data processing system 100, multiple automatedprocesses can take over to evaluate, classify, route, combine andotherwise create views in many dimensions of the data. The emergent dataprocessing system 100 can then use its delivery mechanisms, such as webrecommendations, mobile recommendations, email recommendations, andpersonal catalogs, to name a few, to call out new sets of data that maybe of interest to a user, or to augment and improve a set of data thatis being provided or shown to the user. The default operations of theemergent data processing system 100 can follow the notion that thesystem is a single, dynamic, integrated system that can automaticallyattempt to improve its understanding of the data unless it is otherwiseinstructed not to do so.

Every item that is integrated into the emergent data processing system100 can be compared against any or all other items in the system,showing its relative position to the other items with respect tomeasurements such as quality, theme, content value, sentiment,contribution level, and the like.

The emergent data processing system 100 can create various summaries foreach item that is integrated. These summaries can be event orientedand/or content oriented. Event oriented summaries can be based on theactions of data types such as users. For example, the basis of an eventoriented summary can be the likelihood of a user to purchase at higheror lower price points or be interested in specific brands or quality ofproducts. Content oriented summaries can be based on the quality orgeneral relative position of items. For example, the basis of a contentoriented summary can be the price points of an item relative to similaritems or the quality rating of a brand relative to similar brands.

The emergent data processing system 100 can select and use connectednetworks (e.g., sub-graphs) of content. This capability can be used invarious areas, such as: the use of social networks to influence thegraphs the networks are interconnecting with, and the use of latentnetworks, constructed automatically by the emergent data processingsystem 100, to compute network characteristics that are presentingthemselves indirectly. Social networks influence the graph by allowingthe characteristics and attributes of one person in the network to beadded to or increased in importance to the other people in the network,and to produce specific recommendations that are for example restrictedto the other people in a shared network. Examples of latent networksinclude interest networks, influencer networks, trust networks, andreputation networks. These networks, unlike social networks that arespecifically formed manually by the user, are computed versions based oninterests, where the top N users that have the specific interests (forexample 10) are used to form the network. Interests are tracked peruser, and a latent network can then be produced based on any definition,which is called an interest network. In addition, by tracking otherdata, such as the number of people following a particular person, atrust network can be formed. Then, if the user being followed is writinghis or her content, a reputation network can be formed. An influencernetwork can be formed based on the purchases made that match thecollective purchases in an interest network.

FIGS. 3A and 3B are each a diagram that illustrates an example of ahyper-graph that is used for handling content in the emergent dataprocessing system. Referring to FIG. 3A, there is shown an example of aportion of a hyper-graph structure that is used by the emergent dataprocessing system 100 to handle content. The portion shown includesthree nodes (A, B, and C) and three edges connecting the nodes. Thenodes can represent content (at any granularity level) such ascustomers, users, products, items, themes, catalogs, collections,sub-graphs, or the like, while the edges can represent the relationshipsthat exist between the nodes.

Referring to FIG. 3B, the emergent data processing system 100 canreceive content, can analyze and process the content, and can thenintroduce the content into the existing hyper-graph structure. Theemergent data processing system 100 can identify the relationshipbetween the content and can determine how to introduce the content intothe existing hyper-graph structure. For example, FIG. 3B showsadditional nodes D1, D2, and D3, and corresponding edges between node D1and nodes A and C, between node D2 and nodes A and B, and between nodeD3 and nodes D1 and B.

FIGS. 3A and 3B are presented by way of illustration and not oflimitation. These figures provide an example of how to the emergent dataprocessing system 100 creates a hyper-graph structure with the contentit receives rather than using a typical field-based data warehousingapproach that is generally applied by other systems. Moreover, theexamples shown in FIGS. 3A and 3B illustrate a small hyper-graphstructure with few dimensions. The hyper-graph structure can beconsiderably larger than the one illustrated, having a large number ofdimensions.

FIG. 4 is a diagram that illustrates examples of functional modules inthe emergent data processing system. Referring to FIG. 4, the emergentdata processing system 100 described above can have multiple functionsor services that can be implemented in separate but connected modules.In one implementation, the emergent data processing system 100 can havesix modules: a recommendations and analytics module 410, apersonalization formula module 420, a data unification and integrationmodule 430, a personal catalogs module 440, a competitive analysismodule 450, and a vertical systems module 460.

These modules can be hosted within a single, unified system that caninclude the appropriate servers, operations, integration, and anydevelopment. The applications related to the emergent data processingsystem 100 can run on various platforms, including but not limited tomobile, web, and email. In the operation of these modules, data isintegrated by the emergent data processing system 100 to ensure that themost appropriate recommendations and personalizations are provided. Thedata can include company or retailer data as well as data from otherusers of the emergent data processing system 100.

With respect to the recommendations and analytics module 410, theemergent data processing system 100 can start with a piece of inputdata. The data can be related to a product, but it could just as easilybe any other data, such as a search term, browsing history, or eveninformation about the customer, for example. The emergent dataprocessing system 100 can process and correlate the data based on a widerange of factors to generate a relevant recommendation. The resultingrecommendation can also be a product but need not be so limited. Therecommendation can just as easily be an article, image, catalog, recipe,question, answer, and/or video. For example, recipes indicate what kindsof foods go together, which can indicate what cooking tools areappropriate for those kinds of foods. Images can indicate that there arelogical groupings of products and subjects by virtue of those productsand subjects being together in the same image. Third parties like techmedia websites that publish news, articles, blogs, and podcasts ontechnology and consumer electronics, can provide data that someone hasmanually linked or tagged, connecting specific accessories to mainproducts. If there is a connection or expression between this data anddata in the system, the emergent data processing system 100 can takeadvantage of it.

The recommendations and analytics module 410 of the emergent dataprocessing system 100 can be configured to make recommendations from avariety of perspectives, including similar or closely related items,items frequently bought or viewed together, or accessories to the inputproduct. The emergent data processing system 100 can produce manydifferent types of primary recommendations and a large number of customvariations, which it can deploy based on the user's current context.Once generated by the recommendations and analytics module 410, therecommendations can be filtered using many techniques, includingmerchant black-listing by theme, or through the analysis of sentiment inthe content. In addition to creating recommendations, therecommendations and analytics module 410 can provide the analytics andreporting needed to understand how the recommendations are performingand what needs attention. In this regard, the recommendations andanalytics module 410 can be operable to track each recommendation,click, formula, marketing campaign, and order, for example. Therecommendations and analytics module 410 can then use the data that iscollected and analyzed to help marketing, merchandising and financialanalysts understand what is going on and where there may be additionalopportunities.

The personalization formula module 420 can be used in search, browse,email, and catalog applications, for example. Any application where theinterests and preferences of the individual can be taken intoconsideration can use the results, analysis, functions, and/oroperations of the personalization formula module 420. The emergent orlearning functionality of the personalization formula module 420 enablesan understanding of a customer at many levels of granularity. Theselevels can include individual levels, as a member of multiple cohorts(with their crowd-sourcing benefits), as well as any special groups thatare latently or directly associated with the customer. In this process,the personalization formula module 420 can identify low level purchaseand behavior patterns to determine the probability of activity andsubject interests. In addition, while the emergent data processingsystem 100 looks to optimize recommendations for the customer, it isalso balancing the recommendations against business goals, such asmargin, revenue, competitive positioning, new customer acquisition,customer retention, and/or dormant customer activity, for example.

The emergent data processing system 100 can also address the needs ofmerchants to target customers with their products. This can be achievedby solving the connections that deliver the right customers when thebusiness identifies product objectives or the right products when thebusiness identifies customer objectives. The system can compute andconsider data about age, family, and/or other factors without having toask the customer. The emergent data processing system 100 can balancethe intersection of this data and the appropriate directives to producethe best recommendations at the most appropriate price to the customer.Accordingly, blanket offers and discounts that typically lower prices tounsustainable levels can be eliminated. The emergent data processingsystem 100 can compute from the bottom up, looking at what the customerwants, what the company has, and how it compares against the competitionenabling the emergent data processing system 100 to generate and providerecommendations for the best products that deliver the customer's needsat the best price and margin for the company.

With respect to the data unification and integration module 430, theemergent data processing system 100 can analyze and process a wide rangeof data dimensions. The data can be collected, organized, abstracted,and/or generalized, resulting in the emergent data processing system 100being capable of handling much richer forms of recommendation. Extensivedata, from many sources, can be used by the emergent data processingsystem 100 to fully understand and capitalize on the situations in whichthe various users operate. This unification approach allows the emergentdata processing system 100 to push to a next level of services thatother systems may not be able to provide.

Corporate data can typically be kept in silos and can constantly change.Attempts by other systems to create master ontologies or other top-downstatic views of the data can be ineffective. The emergent dataprocessing system 100 is operable to handle the reality that the datacan be imprecise, full of noise, constantly evolving, and routinelysporadic, missing and incorrectly documented. Data can be difficult tocombine and does not stay consistent, despite what a typical system thatuses rigid data warehousing techniques may promise. The emergent dataprocessing system 100 can address and handle this issue by placing datainto a graph-like data structure (e.g., a hyper-graph type structure)that is abstracted and semantically generalized to enable naturalconnections in the data to occur. This allows applications to work withcontent by expressing what is required indirectly (not having to exactlyspecify what is being search for or added to the graph, as other theexact wording involved is not required to still find ways to match data)and letting the emergent data processing system 100 translate that intothe most appropriate selections and processes.

The emergent data processing system 100 is configured to reduce theambiguity present in the content, while concurrently incorporating themulti-dimensional data into a wide variety of applications. The emergentdata processing system 100 can handle and use a wide range of data typesand sources. Many different types of data can be used to retrieverecommendations and can also be used as recommendations, for exampleproducts, videos, images, blogs, reviews, questions, answers and tasks.Data can be filtered and tagged in multiple dimensions, based not onlyon what is actually present in the content but also based on what isindirectly expressed or learned externally. This includes the ability torelate data to activities, create latent groups of experts, recommendpeople into groups, and manage data indirectly through themes andsentiment. The emergent data processing system 100 can supplyapplication programming interfaces (APIs) that can be incorporated intoapplications and campaigns as they are processed inside the emergentdata processing system 100, or are called externally and used inthird-party applications.

With respect to the personal catalogs module 440, the emergent dataprocessing system 100 can provide catalog-building functionality. Thesystem can be constantly constructing a personal catalog for each user,collection of users, campaigns or any other target desired, collectingand displaying artifacts (e.g., products, items) that the user is likelyto be interested in, artifacts that may have add-on value to somethingthey have already purchased, and/or artifacts that can be related toactivities that the emergent data processing system 100 determines theuser may like or subjects they may have an interest in. These catalogscan be “buy-catalogs” in which the company or retailer can get out infront of the user, understanding and anticipating what the user mayneed, and giving the user what they may want to buy instead of thetraditional user search and browse paradigm forcing the user to findwhat they want. Each of the recommendation techniques supported by theemergent data processing system 100, or any internal or external list ofdata, can be used to generate a personal catalog which can do one ormore of organizing a history of a customer's interaction, providingdeeper recommendations, and using agents to keep up with changes orprice targets.

When organizing a history of the customer's interaction with a retailenvironment, personal catalogs can provide a snapshot of the customer'sactions over time. As a result, even though a customer may not haveacted when an event was occurring, the personal catalog can allow themto go back and reconnect with a point in time where they can reengage inthat area covered by the personal catalog.

With respect to deeper recommendations, each recommendation that ismade, in various selections of combinations, can be turned intocatalogs, both large and small by the personal catalogs module 440 ofthe emergent data processing system 100. Even without actual campaigns,data can be fed into the system to direct the construction of catalogs,and each catalog can be linked into a master catalog for that customer,compiling a table of contents and a back of book style index so thecustomer can easily browse a collection of catalogs (e.g., events). Thecustomer can also share the catalog with others and make notes in it.The customer can purchase directly from the catalog, taking them rightto the checkout page. In this regard, the emergent data processingsystem 100 can call the retailer's add-to-cart and then transfer thecustomer to the retailer's checkout page.

Agents can be an opt-in capability for each catalog generated, whichthen allows the emergent data processing system 100 to communicate in amore personal style with the customer whenever changes (e.g., product,price) occur inside their catalog of interest. The communication can bemore personal because the agent knows the customer, the productsinvolved, the current context, and the current event. This allows theagent to customize the messaging instead of using genericone-size-fits-all messages. The emergent data processing system 100 canreevaluate the catalog and makes updates (e.g., daily, weekly) as neededwith product deletions, additions, price changes, and the like. Theemergent data processing system 100 can communicate with the customer ona weekly basis or as needs dictate

With respect to the competitive analysis module 450, the emergent dataprocessing system 100 is operable to do relative competitor analysis. Acustomer's purchase can be a zero-sum transaction in that a purchasefrom one retailer means that the customer is not buying from otherretailers. No personalization system can be truly successful withoutunderstanding that the decisions are as much about understanding thecustomer and the product as they are about understand the competitorsand their comparative pricing and content quality. It is thereforerelevant to look at every recommendation relative to how it stacks upagainst the competition should the customer be comparing products and/orpricing. The emergent data processing system 100 can compare a productfrom a client retailer against one or more products from competingretailers. The emergent data processing system 100 can base its analysisin the competitor's pricing, quality of product collateral descriptions,and general coverage of similar items (assortment) when matching andcomparing the same or similar products.

The functionality provide by the competitive analysis module 450 neednot be limited to the traditional approaches of product matching thattypically use information such as brand and model or universal productcode (UPC) type. Instead, the emergent data processing system 100 canuse not only that kind of information but also other availableinformation, enabling the emergent data processing system 100 to makemore informed decisions and to work with products individually and insets to make the most appropriate determination as to what is theposition of a client retailer's product relative to the competition.This enables the retailer using the emergent data processing system 100to minimize competitive disadvantages and to craft a winning strategywhen there is comparison shopping.

The emergent data processing system 100 can use the comparative pricingand product quality results to inform its recommendations. In addition,the emergent data processing system 100 is not limited to implementingand using this technique just on the analysis of competitor's products.The emergent data processing system 100 can apply this technique toother types of data, for example, reviews, blogs, videos, pictures,activities, questions, and/or answers. When products can be evaluatedrelative to external content and filtered for positive sentiment, suchinformation can be analyzed and used by the emergent data processingsystem 100 when generating recommendations and user catalogs. Theinformation produced from the competitive analysis can also be used toalter the order of recommendations when external content is deemednegative or when the customer is determined to be one that is sensitiveto price or content.

The vertical systems module 460 can be used in, for example, consumableproducts. A recipe management system is an example of a verticalapplication that can be implemented in the emergent data processingsystem 100. Such system can use information in the emergent dataprocessing system 100 to read and understand recipes, their ingredients,cooking instructions, cooking tools needed, and to determine whichproducts (e.g., in addition to the actual food products) can be ofinterest to the customer that is intended to make the recipe. Theemergent data processing system 100 can provide information to thecustomer to help shop for consumables and cooking tools, determine whatis needed and missing, and what other recipes could be made from anylist of food products. For example, multiple recipes, each with multipleingredients, can enables the emergent data processing system 100 todetermine the kind of cooking the user may be doing, what ingredientsmight be needed, and also what cooking tools are likely needed. Inaddition, the emergent data processing system 100 can analyze personaland cohort repurchase dates to make predictions of what else the cohortsare using and the current customer is not.

FIG. 5 is a diagram that illustrates an example of a personalizationformula used by the emergent data processing system. Referring to FIG.5, there is shown an implementation of the functionality associated withthe personalization formula module 420 described above with respect toFIG. 4. In this implementation, different factors can be considered inmaking a personalization analysis. Such factors can include context 500,customer intelligence 510, business rules 520, and product rankings 530.

The context 500 can include individual aspects 505 such as event, time,location, and/or history, for example. The customer intelligence 510 caninclude individual aspects 515 such as interests, preferences, purchasepoint, and/or quality, for example. The business rules 520 can includeindividual aspects 525 such as dynamic combination of finance,marketing, and/or merchant requirements, for example. The productrankings 530 can include individual aspects 535 such as brand,marketplace, localization, and/or competition related to price andquality, for example.

The information in the context 500, the customer intelligence 510, thebusiness rules 520, and the product rankings 530 can be combined toproduce a recommendation 540 that includes individual aspects 545 suchas product, weight, price, discount, membership point, and/orrequirements, for example. Any computations or formulas can considermany relationships in the graph, such as the importance of revenuerelative to margin, the importance of acquiring new customers relativeto revenue, the importance of a customers current interests relative totheir previous interests and/or to price reductions, the importance ofthe popularity in general of the product relative the more specificmatch to the customers interests, and/or the importance of recommendingproducts where the company has a better assortment and pricing than thecompetitors. These comparisons can be made by producing weighted valuesto determine an appropriate recommendation.

FIG. 6 is a screenshot that illustrates an example of personal cataloggenerated by the emergent data processing system. Referring to FIG. 6,there is shown an implementation of the functionality associated withthe personal catalogs module 440 described above with respect to FIG. 4.In this implementation, a screenshot 600 of a personal catalog can begenerated by the emergent data processing system 100 to highlight aparticular type of lawn mower and other related accessories that may beof interest to a particular customer. The layout and/or the informationpresented in the screenshot 600 are provided by way of illustration andnot of limitation.

There are various aspects for consideration when generating a personalcatalog using the emergent data processing system 100. For example,certain sections of the personal catalog can be of importance and can beidentified as such, the personal catalog can be automatically generated,historical snapshots may be kept, the personal catalog may be organizedby major product verticals, by activities, and/or by subjects, indexesand table of contents can be created, the content inside the personalcatalog can be personalized, and different aspects of therecommendations inside the personal catalog can be tracked such as thosethat have new prices, replacement products, and/or are trending popularright now. Another aspect can be to create a personal catalog byproviding input or some other information to the personal catalog ofwhat the user is interested in. Moreover, another aspect can be toeasily find variants of an item and update the personal catalog withsome or all of the variants.

FIGS. 7A-7C are each a screenshot that illustrates additional examplesof personal catalogs generated by the emergent data processing system.These figures show an implementation of the functionality associatedwith the personal catalogs module 440 described above with respect toFIG. 4. Referring to FIG. 7A, there is shown a screenshot 700 of apersonal catalog that includes new products or items that the emergentdata processing system 100 has identified as being of possible interestto a customer. FIG. 7B shows a screenshot 710 of a personal catalog thatincludes an alphabetical index of items available to the customer in thepersonal catalog. FIG. 7C shows a screenshot 720 of a personal catalogthat includes related products or items. For example, screenshot 720 canprovide tools or hardware that is related to yard work given theinterest the customer may have on lawn mower as illustrated by thescreenshot 600 shown in FIG. 6. The layout and/or the informationpresented in the screenshots 700, 710, and 720 are provided by way ofillustration and not of limitation.

FIG. 8 is a diagram that illustrates touch points and feedback in theemergent data processing system.

FIG. 9 is a diagram that illustrates an example of privacy policies inthe emergent data processing system. Referring to FIG. 9, there is showna system 900 that uses an emergent data processing system (e.g., TEC) toprovide privacy operations. To generate personalized recommendationsbetween a company or retailer and third-party systems, the emergent dataprocessing system may not be able to pass personal information to thethird-party system because of privacy policies regarding the use ofpersonal data. Instead of using personal data, other information can beintegrated such as product data and/or cohort data. The cohort data canbe limited to include a summation of personal data to adhere to theprivacy policies. By using the approach described above, the emergentdata processing system can properly compute the recommendations up tothe individual level.

The emergent data processing system can have code resident on thethird-party systems that can access personal data during the last stepsof processing and apply results to the recommendations returned from theemergent data processing system based on cohorts. This approach enablesthe generation of personalized results but within the privacy policiessupported by the parties involved. These servers work by calling theemergence data processing system in a generic way, without providingspecific customer information. Then, upon receiving the data back fromthe emergent data processing system, the third-party system can applypersonal adjustments to the data, either by using its own personal data,or by having personal computation algorithms from the emergent dataprocessing system resident on the third party system, and executing thisphase on those servers. In this way, the primary (host) server does notneed to see personal data.

FIGS. 10A-10C are each a diagram that illustrates examples ofperformance benchmarks produced to evaluate the emergent data processingsystem. Referring to FIG. 10A, there is shown a graph 1000 thatillustrates results from testing of an implementation of the emergentdata processing system 100 (e.g., TEC) against other systems (e.g.,non-TEC). The tests included email-based product recommendations acrossa wide range of products. Graph 1000 provides conversion results (thenumber of orders factored against the number of emails sent), which showas good or better results than the conversion provided by othertechniques.

Referring to FIG. 10B, there is shown a graph 1010 that also illustratesresults from testing of an implementation of the emergent dataprocessing system 100 (e.g., TEC) against another system (e.g.,non-TEC). Graph 1010 provides revenue-per-email (RPE) results, whichshow as good or better results than the RPE provided by the competitor.

Referring to FIG. 10C, there is shown a graph 1020 that also illustratesresults from testing of an implementation of the emergent dataprocessing system 100 (e.g., TEC). Graph 1020 provides combinedconversion and RPE results for the emergent data processing system 100.

FIGS. 11A and 11B are each a diagram that illustrates additionalexamples of performance benchmarks produced to evaluate the emergentdata processing system. Referring to FIG. 11A, there is shown a graph1100 that illustrates results from testing of an implementation of theemergent data processing system 100 (e.g., TEC). These tests includedemail promotions campaigns in which the performance of the emergent dataprocessing system 100 was compared with products selected by merchantsbased on their perspective and experience. The results show betteroutcomes when the emergent data processing system 100 is used inconnection with unique click rates, conversion rated, average ordervalue, and revenue-per-email.

Referring to FIG. 11B, there is shown a graph 1110 that also illustratesresults from testing of an implementation of the emergent dataprocessing system 100 (e.g., TEC). Graph 1110 provides tests resultsfrom an email campaign that shows that the emergent data processingsystem 100 (e.g., TEC) is capable of generating higher revenue thanother, non-emergent technology systems (e.g., non-TEC).

FIGS. 12-14 are each a flow diagram that illustrates examples of stepsfor data handling by the emergent data processing system. Referring toFIG. 12, there is show a flow chart 1200 in which, at step 1210, theemergent data processing system 100 described above can receive contentsuch as customer or user data and/or product data, for example. At step1220, the emergent data processing system 100 can store the content intoa hyper-graph structure such as the one illustrated in FIGS. 3A and 3B.At step 1230, the emergent data processing system 100 can select aportion of the hyper-graph structure that corresponds to a particularcustomer or user. At step 1240, the emergent data processing system 100can base a recommendation for a product or some other item based on thecontent in the selected portion of the hyper-graph structure.

Referring to FIG. 13, there is shown a flow chart 1300 in which, at step1310, the emergent data processing system 100 described above canreceive content such as customer or user data and/or product data, forexample. At step 1320, the emergent data processing system 100 can storethe content into a hyper-graph structure such as the one illustrated inFIGS. 3A and 3B. At step 1330, the emergent data processing system 100can select a portion of the hyper-graph structure that corresponds to aparticular customer or user. At step 1340, the emergent data processingsystem 100 can generate a personal catalog for the customer or userbased on the content in the selected portion of the hyper-graphstructure.

Referring to FIG. 14, there is show a flow chart 1400 in which, at step1410, the emergent data processing system 100 described above canreceive content such as customer or user data and/or product data, forexample. At step 1420, the emergent data processing system 100 can storethe content into a hyper-graph structure such as the one illustrated inFIGS. 3A and 3B. At step 1430, the emergent data processing system 100can select a portion of the hyper-graph structure that corresponds to aparticular customer or user. At step 1440, the emergent data processingsystem 100 can compare data for products from different sources (e.g.,different retailers). the At step 1450, the emergent data processingsystem 100 can base a recommendation for a product or some other itembased on the content in the selected portion of the hyper-graphstructure and on the results from the comparison made in step 1440.

Another implementation provides for a non-transitory machine and/orcomputer readable storage and/or media, having stored thereon, a machinecode and/or a computer program having at least one code sectionexecutable by a machine and/or a computer, thereby causing the machineand/or computer to perform the steps as described herein for emergentdata processing.

Accordingly, the present disclosure can be realized in hardware,software, or a combination of hardware and software. The presentdisclosure can be realized in a centralized fashion in at least onecomputer system; or in a distributed fashion where different elementsare spread across several interconnected computer systems. Any kind ofcomputer system or other apparatus adapted for carrying out the methodsdescribed herein is suited. A typical combination of hardware andsoftware may be a general-purpose computer system with a computerprogram that, when being loaded and executed, controls the computersystem such that it carries out the methods described herein.

The present disclosure can also be embedded in a computer programproduct, which includes all the features enabling the implementation ofthe methods described herein, and which when loaded in a computer systemis able to carry out these methods. Computer program in the presentcontext means any expression, in any language, code or notation, of aset of instructions intended to cause a system having an informationprocessing capability to perform a particular function either directlyor after either or both of the following: a) conversion to anotherlanguage, code or notation; b) reproduction in a different materialform.

While the present disclosure has been described with reference tocertain implementations, it will be understood by those skilled in theart that various changes may be made and equivalents may be substitutedwithout departing from the scope of the present disclosure. In addition,many modifications may be made to adapt a particular situation ormaterial to the teachings of the present disclosure without departingfrom its scope. Therefore, it is intended that the present disclosurenot be limited to a particular implementation disclosed, but that thepresent disclosure will include all implementations falling within thescope of the appended claims.

What is claimed is:
 1. A method, comprising: in a system having at leastone server that is operable to handle retail data and to communicateelectronic messages with at least one customer device, the at least oneserver comprising memory configured to maintain a hyper-graph structure,receiving content from a first source of data, the content including aplurality of components of customer data and product data, wherein oneor more of the components comprise data in one or more undocumentedfields; storing, in the memory of the at least one server, the pluralityof components of the received content to establish interconnections inthe hyper-graph structure of the received data with a plurality ofcomponents of content for related customer data and product data fromother sources, wherein the data in the one or more undocumented fieldsare placed unaltered in the hyper-graph structure, and the data from theone or more undocumented fields are understood via theirinterconnections in the hyper-graph structure; configuring theinterconnections to dynamically connect at various levels of granularityand interpretation; filling in particular components missing from dataof a first customer in the hyper-graph structure using known values ofthe particular components in data of a second customer in thehyper-graph structure, based on an amount of intersection of othercomponents of the data of the first customer and the second customer,wherein the known values are weighted to indicate that the filled-incomponents are characteristic of the first customer, and wherein theknown values are weighted to indicate that the filled-in components arenot characteristic of the first customer; selecting a portion of thehyper-graph structure based on similarity of components of data for aparticular customer to components of customer data present in thehypergraph structure; performing a vertical analysis of consumableproducts of the selected portion of the hyper-graph structure;constructing a personal catalog from at least one recommendation for theparticular customer based on the content in the selected portion of thehyper-graph structure, for transmission to a device of the particularcustomer for viewing; and providing a graphical user interface thatcomprises one or more graphical elements that can be selected toorganize and display the personal catalog based on the performedvertical analysis.
 2. The method of claim 1, wherein the recommendationincludes one or more of a product, an article, an image, a catalog, arecipe, a question, an answer, and a video, or wherein the selectedportion of the hyper-graph structure corresponds to a cohort ofcustomers that includes the particular customer.
 3. The method of claim1, comprising filtering the recommendation based on one or both of amerchant black-listing by theme and sentiment information about theparticular customer.
 4. The method of claim 1, comprising generating therecommendations for the particular customer based on business parametersstored in the at least one server, the business parameters including oneor more of margin, revenue, competitive positioning, costumeracquisition, customer retention, and customer activity.
 5. The method ofclaim 1, wherein the first customer is one of a plurality of customers,wherein the plurality of customers are characterized by multiplepersonas, wherein each persona is based on characteristics and data fromthe plurality of customers, wherein the system determines which of themultiple personas is prevailing at a particular time or for a particularinteraction for the first customer, and wherein the prevailing personais based on characteristics and data from the plurality of customersincluding the first customer.
 6. The method of claim 1, wherein thecustomer data or product data includes data of different formats, themethod comprising: normalizing the customer data or the product datathrough abstraction and semantic generalization; and afternormalization, storing the customer data or the product data into thehyper-graph structure.
 7. The method of claim 1, wherein one or both ofthe first customer and the second customer can be defined in an emergentdata processing system using dynamic data dimensions.
 8. The method ofclaim 1, comprising: generating an electronic message that includes therecommendation; and tracking an interaction of the particular customerwith the recommendation in the electronic message.
 9. A method,comprising: in a system having at least one server that is operable tohandle retail data and to communicate electronic messages with at leastone customer device, the at least one server comprising memoryconfigured to maintain a hyper-graph structure, receiving content from afirst source of data, the content including a plurality of components ofcustomer data and product data, wherein one or more of the componentscomprise data in one or more undocumented fields; storing the pluralityof components of the received content to establish interconnections inthe hyper-graph structure of the received data with a plurality ofcomponents of content for related customer data and product data fromother sources, wherein the data in the one or more undocumented fieldsare placed unaltered in the hyper-graph structure, and the data from theone or more undocumented fields are understood via theirinterconnections in the hyper-graph structure; normalizing, in thememory of the at least one server, the plurality of components of thereceived content to establish interconnections in the hyper-graphstructure of the received data with a plurality of components of contentfor related customer data and product data from other sources;configuring the interconnections to dynamically connect at variouslevels of granularity and interpretation; filling in particularcomponents missing from data of a first customer in the hyper-graphstructure using known values of the particular components in data of asecond customer in the hyper-graph structure, based on an amount ofintersection of other components of the data of the first customer andthe second customer, wherein the known values are weighted to indicatethat the filled-in components are true for the first customer, andwherein the known values are weighted to indicate that the filled-incomponents are not true for the first customer; selecting a portion ofthe hyper-graph structure based on similarity of components of data fora particular customer to components of customer data present in thehypergraph structure; performing a vertical analysis of consumableproducts of the selected portion of the hyper-graph structure;constructing a personal catalog from at least one recommendation for theparticular customer based on the content in the selected portion of thehyper-graph structure, for transmission to a device of the particularcustomer for viewing; and providing a graphical user interface thatcomprises one or more graphical elements that can be selected toorganize and display the personal catalog based on the performedvertical analysis.
 10. The method of claim 9, wherein the personalcatalog includes historical interaction information of the particularcustomer.
 11. The method of claim 9, wherein the personal catalogincludes a recommendation that includes one or more of a product, anarticle, an image, a catalog, a recipe, a question, an answer, and avideo.
 12. The method of claim 11, comprising: determining one or moreitems related to the recommendation; and providing the one or more itemsin the personal catalog.
 13. The method of claim 9, comprising linkingthe personal catalog to one or more additional catalogs corresponding tothe particular customer.
 14. The method of claim 9, wherein the selectedportion of the hyper-graph structure corresponds to a cohort ofcustomers that includes the particular customer.
 15. A method,comprising: in a system having at least one server that is operable tohandle retail data and to communicate electronic messages with at leastone customer device, the at least one server comprising memoryconfigured to maintain a hyper-graph structure, receiving content from afirst source of data, the content including a plurality of components ofcustomer data and product data, wherein one or more of the componentscomprise data in one or more undocumented fields; storing, in the memoryof the at least one server, the plurality of components of the receivedcontent to establish interconnections in the hyper-graph structure ofthe received data with a plurality of components of content for relatedcustomer data and product data from other sources, wherein the data inthe one or more undocumented fields are placed unaltered in thehyper-graph structure, and the data from the one or more undocumentedfields are understood via their interconnections in the hyper-graphstructure; configuring the interconnections to dynamically connect atvarious levels of granularity and interpretation; filling in particularcomponents missing from data of a first customer in the hyper-graphstructure using known values of the particular components in data of asecond customer in the hyper-graph structure, based on an amount ofintersection of other components of the data of the first customer andthe second customer, wherein the known values are highly weighted toindicate that the filled-in components are characteristic of the firstcustomer, and wherein the known values are lowly weighted to indicatethat the filled-in components are not characteristic of the firstcustomer; selecting a portion of the hyper-graph structure based onsimilarity of components of data for a particular customer to componentsof customer data present in the hypergraph structure; performing avertical analysis of consumable products of the selected portion of thehyper-graph structure; comparing data stored in the at least one servercorresponding to one commercial entity with data stored in the at leastone server corresponding to another commercial entity, the data beingcompared including product data; constructing a personal catalog from atleast one recommendation for the particular customer based on thecontent in the selected portion of the hyper-graph structure and thecomparison, for transmission to a device of the particular customer forviewing; and providing a graphical user interface that comprises one ormore graphical elements that can be selected to organize and display thepersonal catalog based on the performed vertical analysis.
 16. Themethod of claim 15, wherein the comparison is based on one or both ofproduct pricing and product quality.
 17. The method of claim 15,comprising comparing data stored in the at least one servercorresponding to one commercial entity with data stored in the at leastone server corresponding to another commercial entity, the data beingcompared including one or more of product data, a review, a blog, avideo, a picture, an activity, a question, and an answer.
 18. The methodof claim 15, comprising: generating an electronic message that includesthe recommendation; and tracking an interaction of the particularcustomer with the recommendation in the electronic message.
 19. Themethod of claim 15, wherein the selected portion of the hyper-graphstructure corresponds to a cohort of customers that includes theparticular customer.
 20. The method of claim 15, wherein each of thecustomer data and the product data includes data of different formats,the method comprising: normalizing the customer data and the productdata through abstraction and semantic generalization; and afternormalization, storing the customer data and the product data into thehyper-graph structure.
 21. The method of claim 1, wherein the verticalanalysis of the consumable products provides, in the personal catalog,recommendations for tools that can be used in connection with theconsumable products.
 22. The method of claim 9, wherein the at least onerecommendation is based on at least two characteristics of the data ofthe content in the selected portion of the hyper-graph structure: asimilarity connection and a semantic connection, wherein the semanticconnection is not based on similarity.
 23. The method of claim 15,wherein the customer data components comprise one or more of aninterest, a probable activity, a behavior, and a specific attribute of acustomer, and the product data comprises one or more of a property, anattribute, and a characteristic of a product item.